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Abstract 

We consider a variation of the spectral sparsification problem where we are required to keep 
a subgraph of the original graph. Formally, given a union of two weighted graphs G and W 
and an integer fc, we are asked to find a fc-edge weighted graph Wk such that G + Wk is a 
good spectral sparsifer of G + W . We will refer to this problem as the subgraph (spectral) 
sparsification. We present a nontrivial condition on G and W such that a good sparsifier exists 
and give a polynomial time algorithm to find the sparsifer. 

As a significant application of our technique, we show that for each positive integer k, every 
n- vertex weighted graph has an (n — l + fc)-edge spectral sparsifier with relative condition number 
at most ^ log 71 (log log ?i) where 0{) hides lower order terms. Our bound is within a factor of 
O(loglogn) from optimal. This nearly settles a question left open by Spielman and Teng about 
ultrasparsifiers, which is a key component in their nearly linear-time algorithms for solving 
diagonally dominant symmetric linear systems. 

We also present another application of our technique to spectral optimization in which the 
goal is to maximize the algebraic connectivity of a graph (e.g. turn it into an expander) with a 
limited number of edges. 
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1 Introduction 



Sparsification is an important technique for designing efficient graph algorithms, especially for dense 
graphs. Informally, a graph (5 is a sparsifer of G if they are similar in a particular measure (which is 
important to the application that one has in mind), and that G has linear or nearly linear mimber 
of edges. Various notions of graph approximation and sparsification have been considered in the 
literature. For example. Chew's [6] spanners (for shortest path planning) have the property that 
the distance between every pair of vertices in G is approximately the same as in G. Benczur and 
Kargcr's [4] cut-sparsificrs (for cuts and flows) have the property that the weight of the boundary 
of every set of vertices is approximately the same in G as in G. 

In this paper, we will mainly be interested in the spectral notion of graph similarity introduced 
by Spielman and Teng [18], [20]: we say that a weighted undirected graph ii" is a K-approximation 
of another G if for all x G R^, 

CqX < X^CqX < KX^ Cqx (1) 

where for a weighted undirected graph G, Cq is the Laplacian matrix of G defined as the following: 
For each CG{i-,i) is equal to the sum of weights of all edges incident to vertex i and for i ^ j, 
^oihj) = ^'^-"i,?) where Wi_j is the weight on edge (?',j). 

In [18, 20], the following spectral sparsification problem is considered. Given a weighted graph 
G = {V,E,w), an integer rh < \E\, and k > 1, find a graph G = {V,E,w} such that \E\ < rh 
and G is a ^-approximation of G. We will refer to this problem and its corresponding optimization 
problem as the Spectral Sparsification. Spielman and Teng showed that every weighted graph 
has a nearly linear-sized spectral sparsifier and gave a nearly linear-time algorithm for computing 
such a sparsifier. Recently, Batson, Spielman, and Srivastava [3] gave a beautiful, polynomial-time 
construction to produce a linear-sized spectral sparsifier. 

In this paper, we introduce a variation of the spectral sparsification problem which we will refer 
to as the Subgraph Sparsification. In our version, we are given two weighted graphs G and 
W, an integer k and k>1. The goal is to find a k-edge weighted graph such that (G -|- Wk) is 
a K-approximation of (G + W). The challenge in the new version of the sparsification problem is 
that we have to respect part of the graph, i.e., G, and only modify part of graph given in W. 

As the main technical contribution of the paper, we give a nontrivial condition about G and 
W such that a good sparsifier exists. Our proof critically uses the intuition of Batson, Spielman, 
and Srivastava [3], that uses potential functions that guide an incremental process for selecting 
the edges of the sparisifier. We will refer to that as as the BSS process. We have enhanced their 
approach with new understanding about subspace sparsification and spectral approximation. 

Our challenge, at high level, is the following. The BSS process uses two carefully chosen barriers 
(see Section 2) so that at each step, all eigenvalues can be kept far enough from these barriers. 
They have B(n) edges to select. So they consider the entire n-dimensional space and have step size 
0(l/n) on these barriers. 

On the other hand, we can only add k edges, where k can be arbitrarily smaller than n. 
The addition of each edge can only increase smallest eigenvalue to the second smallest eigenvalue. 
Therefore the addition of k edges can only improve the subspace defined by the k smallest eigenvalue. 
Now, the critical part of the argument is that to build a good sparsifier, we need to ensure that 
the addition of the edges does not increase the high spectra by too much. So in our incremental 
process, we need to keep track of two subspaces, a fixed one defined by the k smallest eigenvalues 
and a floating one defined by the higher spectra. 
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We developed an analysis for performing spectral analysis in the projection of a sequence of two 
subspaces, which might be interesting on its own right. Our analysis also provide a nice example 
for using majorization. 

Our ability to conduct sparsification on a subgraph enables us to obtain improved results for 
a few problems on spectral optimization. The first application that we consider is the problem of 
finding ultrasparsifiers as defined in Spielman and Teng [18]. For parameters k > 1 and k > 1, 
a weighted undirected graph [/ is a {k, k)-ultrasparsifier of another graph G, if U has at most 
n — 1 + k edges, and Cu ^ ^ k ■ £u- Ultrasarsifiers are essential in the application of the 
preconditioning techniques for solving linear systems [18, 20]. It has been shown in [18] that every 
weighted undirected graph G has a log*^'-"'^^ n, /c) ultrasparsifiers, for any k. As a significant 
application of our subgraph sparsification technique, we show that for every positive integer k, 
every n-vertex weighted graph has a {j log n O (log log n), A;)-ultrasparsifier. Our bound is within a 
factor of 0(log log) from the optimal. This new result nearly settles a question about ultrasparsifiers 
left open by Spielman and Teng. 

At high level, our solution to ultrasparsification is quite simple, once we have our subgraph 
sparsification result. Given a weighted graphG, we first construct a low-stretch spanning tree 
[2, 7, 1] T of G. We then apply an elegant result of Spielman and Woo [21] which states that the 
sum of the relative condition numbers of Cq and Ct is equal to the total stretch to embed G onto 
T. We will also use Spielman-Woo's tail distribution bound on the number of relative eigenvalues 
of Cg and Ct that are larger than a given parameter. 

Algorithmically, we start with the best available [1] low-stretch spanning tree T of G whose 
total stretch is nlognO(loglogn). We then consider the subgraph sparsification problem defined 
by T and W = zG. We apply the structure theorem of Spielman and Woo [21] to 

TtlognO(loglogn) Jr-r- j i- l j 

show that (r, W) satisfy our condition for subgraph sparsification and apply our result to show 
that there exists a fc-edge weighted graph Wk whose edges are in W such that T + Wk is a spectral 
approximation of T -|- W. It is then not hard to prove that T + is an a log n 0(log log n), k)- 
ultrasparsifier. 

As another application of our technique on subgraph sparsification, we consider the following 
spectral optimization problem studied in [5]: Given a graph G and a parameter A;, we are asked 
to find k edges amongst a set of candidate edges to add to G so as to maximize its algebraic 
connectivity. Algebraic connectivity has emerged as an important parameter for measuring the 
robustness and stability of a network and is an essential factor in the performance of various 
search, routing and information diffusion algorithms. 

The spectral optimization considered in this paper is known to be NP-hard [15] and no ap- 
proximation guarantee for it was known prior to our work. Wc give an SDP-based approximation 
algorithm for the problem. Our techniques for subgraph sparsification enable us to develop a novel 
rounding scheme in order to find a combinatorial solution. Since the integrality gap of the SDP is 
unbounded, our analysis involves adding a separate upper bound, which is roughly the A;-th largest 
eigenvalue of the Laplacian of G to approximate the optimum solution. 

2 Preliminaries 

Matrix Notation and Definitions. We denote the Laplacian of a graph G by Cq- For brevity, we 

write Gi ^ G2 to denote Cd ^ C,G2- an n x n matrix A, let Amm(^) = •^i(^) < < • • • < 

\n{A) = Ainax(^) be the set of eigenvalues in the increasing order. Let A'^ be the pseudoinverse 
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of A. If A is symmetric, A^ is also symmetric and AA^ = A^A = Piin(A)) where Pini(A) is the 
orthogonal projection on Im(A). Let A* B = iv B be the Frobenius product of matrices A and 
B. Wc define the condition number of a non-singular matrix A as k = ||^|| ||^~^ ||, which is equal 
to Amax(^)/Amin(^) if ^ IS a (symmetric) positive definite matrix. For positive definite matrices 
A, B with Im^ = ImS, we define the relative condition number as 

, . Ax x^Bx 
k( " 



, . X Ax X Bx 

KiA,B) = max „ • max „ 

x^ker B X^ Bx x^ker A X^ Ax 



Ultrasparsifiers. We say that a graph is A:-ultra-sparse if it has at most n — 1 + k edges. We 
note that a spanning tree is 0-ultra-sparse. A (ac, k) ultra- spar sifier of a graph G = (V, E, w) is a 
A;-ultra-sparse subgraph of G such that U :< G ^ k ■ U [18]. 

3 Matrix Sparsifiers 

In this section, we prove an analog of the sparsification theorem of Batson, Spielman, and Srivas- 
tava [3]. 

Definition 3.1. (Graph Patch) Let G be a (weighted) graph. A graph W on the vertices of G is 
a {k,T, X*) -patch for G if the following properties hold) , 

We prove that for every patch, there exists a "patch sparsifier" supported on 0{k) edges. 
Specifically, we prove the following theorem. 

Claim 3.2. Let W = {V,E\Y,{we}eeEw) ^ {k,T, X*)-patch for G with edge weights We and 
N > 8k. Then there is a weighted graph = (V, Ew^., {'"^ejee-Ewfe) ^^^'^ ^^9^ weights We such that 

1. Wk has at most N edges; Ew^ C Ew- 

2. ci min(A''/r, Cq+w ^ ^G+Wk ^ C2£^G+W, for some absolute constants ci and C2. 

3. The total weight of edges, X^eefi^^ ^fe' most min(l, A''/T) ^g^^,^ We- 

We say that is a patch sparsifier of W with respect to G. 

The claim will follow immediately from the following theorem, which is is of independent inter- 
est. We will also show another (related) application of this theorem in Section 5. 

Theorem 3.3. Suppose we are given a positive definite nxn matrix X and a sequence of matrices 
Yi = VivJ (i = 1, . . . ,m) with 

m 

X-\-J2Yi = M*, 

i=l 



^wc have Xk+i{jCG^a+w) — •^fc+i(('^G+w)^^^'^G('^G+VK)'^''^)' since \i{AB) = Xi{BA) for every two square matri- 
ces A and B 
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and Xraax{M*) < 1. Additionaly, suppose each matrix has cost costi and YliLi ''o^'^ti = 1- Let 
A* = Xk+i{X), and T = [tr(M* — X)~\. Then for every N > 8k there exists a set of weights Wi 
with {{wi : Wi 0}\ = N such that the matrix M = X + X^^li WiYi satisfies, 

ci min(7V/r, 1) • A* • A^in(M*) < A^in(M) < A„,ax(M) < C2, 

where ci and C2 are some absolute constants, and X^^i Wicosti < min(l, N/T). 

Proof Overview. Our proof closely follows the approach of Batson, Spielman, and Srivas- 
tava [3]. We construct matrix M in N steps; at each step we choose an index i and weight Wi 

and add WiYi to the sum X + "^^i WiYi. Recall that Batson, Spielman, and Srivastava define two 
"barriers" / and u and maintain the property that all eigenvalues of M lie between / and u. At 
each step, they increase I and u and update matrix M so that this property still holds. Finally, the 
ratio between u and I becomes very close to 1, which means that Amin(Af) is very close to Amax(-A^)- 
During this process, they keep track not only of the smallest and largest eigenvalues of M but of 
all n eigenvalues to avoid accumulation of eigenvalues in neighborhoods of / and u. To this end, 
they define two potential functions, the lower potential function $|(Af) = ^"^^ x {M)-i 

upper potential function $"(M) = Ylii=i u-\i{M) ' ^'^'^ then ensure that ^i{M) and $"(M) do not 
increase over time. That guarantees that all eigenvalues of M stay far away from I and u. 

In our proof, however, we cannot keep an eye on all eigenvalues. After each step, only one 
eigenvalue increases, and thus we need 6{n) steps to increase all eigenvalues participating in the 
definition of <&;(M). But our goal is to "patch" X in roughly k steps. So we focus our attention 
only on k smallest and T largest eigenvalues. 

Let S be the eigenspace of X corresponding to k smallest eigenvalues, and Pg be the projection 
onto S. We define the lower potential function as follows, 

k 

where denotes the restriction of A to the space S {A\g is a. k x k matrix). Note that the space 
S is fixed, and the eigenvector corresponding to the smallest eigenvalue will not necessarily lie in 
S after a few steps. We want to ensure that after N steps, 

m m 

^WiYi\g t cmm{N/T, 1) YX^ls = cram{N/T, 1)(M* - X)]^, 
i=l i=l 

or in other words, \mm{{Z{J2Zi^'iyi)Z)\s) > cmm{N/T,l), where Z = {{Ps{M* - X)Ps)^f^'^ . 
To this end, we show how to update M and / so that ^i{Z{'^J!^^WiYi)Z) does not increase, and 
/ equals cmin(A^/r, 1) after N steps. It remains to lower bound \ram{M) in the entire space. We 
know that all eigenvalues of X (and therefore, of M) in are at least A*. We show that that 
together with an upper bound on Amax(-^) implies that Ainin(-^) ^ ci mm[N/T, 1) • A*Amin(-^*) 
(the product of the lower bounds on Amin in spaces S and divided by the upper bound on Amax)- 
Similarly, we amend the definition of the upper potential function. Since we need to bound Amax 
in the entire space, we cannot restrict $"(M) to a fixed subspace. For a matrix A, we consider 
the eigenspace of A corresponding to its largest T eigenvalues. Denote it by La{A); denote the 
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projection onto L{A) by Pl{A)- Then 



N 



u - XAA) 

i=n-T+l 

Note that both definitions of $"(^4) — in terms of regular inverse and in terms of pseudoinverse — 
are equivalent since L[A) is an invariant subspace of A. However, ^i{A) is not equal to tr(P5(^ — 
lI)~^Ps) in general since S is not necessarily an invariant subspace of A. 

Our algorithm and analysis are similar to those of Batson, Spielman, and Srivastava [3]. How- 
ever, several complications arise because we are controlling eigenvalues in different subspaces and, 
moreover, one of these subspaces, L[A), is not fixed. 

Let us summarize the proof. We construct the matrix M iteratively in A'" steps. Let A^'^^ be the 
matrix and w^f^ be the weights after q steps. We define an auxiliary matrix S(^) as Z{A^'i^ — X)Z. 
We have, 

^(5) =X + Y^ w^^'^Yi- B^i^ = wfzYiZ = Z{A'^i'> - X)Z. 

i i 

We will ensure that the following properties hold after each step (for some values of constants Iq, 
Sl, uq, 5u, tL, which we will specify later). 

1. < eL and < eu- 

2. Each matrix A^'^'' and B^'^^ is obtained by a rank-one update of the previous one: 

^(g+i) ^ ^(5) + ^Y^, S^'^+i) = + tZYiZ 

for some i. 

3. Lower and upper potentials do not increase. Namely, for every = 0, 1, . . . , A/", 

$no+(9+i)5c7(^(5+i)) < $«o+95c/(^W) < and < ^i.+qS^^iB^"'^) < e^. 

4. At each step g, AminC-B^^^I^) > I = h + q^L and AmaxC^^^^) <u = UQ + q5u. In particular, this 
condition ensures that all terms in the definitions of upper and lower potentials are positive. 

5. At each step g, the total cost is at at most q/ max{N,T): Y^w^f^ costi < qj max(Ar, T). 

We present the complete proof in Sections 3.2 and 3.3. In Section 3.2, we first find conditions 
under which we can update and u (Lemma 3.10), and B^'^^ and I (Lemma 3.11). Then we 

show that both conditions can be simultaneously satisfied (Lemma 3.12). In Section 3.1.2, we prove 
several theorems that we need later to deal with a non-fixed subspace J-i{^A). Finally, in Section 3.3, 
we combine all pieces of the proof together. 



3.1 Some Basic Facts about Matrices 
3.1.1 Sherman— Morrison Formula 

We use the Sherman-Morrison Formula, which describes the behavior of the inverse of a matrix 
under rank-one updates. We first state the formula for regular inverse [8], and then we show that 
a similar expression holds for the pseudoinverse. 
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Lemma 3.4 (Sherman-Morrison Formula). If A is a nonsingular n x n matrix and Y = vv^ is a 
rank-one update, then 

Lemma 3.5. If A is a symmetric (possibly singular) nxn matrix, Y = vv^ is a rank-one update, 
then 

where P is the orthogonal projection on Im(^). 

Proof Let v = Pv and Y = PYP = vv^ . Note that A'^YA^ = A^YA\ since PA"^ = P, and 

y = tr A'fY = tr A\PYP) = tr{PA^P)Y = y. 

We need to verify that 

(A + y) L4t — ^ ] = \A^ — ^ ](A-\-Y) = P. 

Since ^ is a symmetric matrix, AA^ = A^A = P. Since P^ = P, PYP = Y and YA^Y = 
vv'^Avv'^ = v{A • Y)v'^ = {A* Y)Y. We calculate, 

1 + AUY 

Similarly, 

(^t_ "^^^f . ){A + Y) = P 



□ 



3.1.2 Majorization 



Lemma 3.6. (Majorization) For every positive semidefinite matrix A, every projection matrix P, 
and every r G {1, . . . , n} 

n n 

J2 Ai(^)> Yl ^iiPAP)- (2) 

j=n— r+1 i=n— r+1 

In particular, Amax(^) > ^ma.^{PAP). 

Proof. Let ei, . . . , be an orthonormal eigenbasis of A so that Cj has eigenvalue \i{A). Similarly, 
let 

ci, ■ ■ ■ , Cn be an orthonormal eigenbasis of PAP so that has eigenvalue \i{PAP^. Write 

n 
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Note that if Xi{PAP) then G lm{PAP) C Iin(P) and Pcj = e^. Then 

n 

\i{PAP) = efPAPsi = CiAci = J^i^^j^^if^M)- 

i=i 

If \i{PAP) = then triviahy 

n 

\i{PAP) = < ^{ej,eifXj{A). 

3=1 

Therefore, 

5^ \^{PAP)< Y.{e,rer?\,{A) = Y,{ {e,,~e^?\xAA). 

i=n—r+l i=n—r+l j=l j=l \i=n—r+l / 

That is, Aj(P^P) is at most the sum of Xj{A) with weights Y17=n-r+ii^j ^ ^if' ■ The total 

weight of all . . . , A„(^) is r: 

n n n 
i=n— r+l j=l i=n— r+1 




The weight of each eigenvalue Aj(^) in the sum is at most 1: 

n n 

Y (ej>ei)2 < ^(ej-,ei)2 = 1. 

i=n—r+l 1=1 

Therefore, the sum does not exceed the sum of the r largest eigenvalues X]"=„_j.+i \r{A). □ 

Corollary 3.7. For every positive semidefinite matrix A, every projection matrix P and u > 
^max{A), the following inequality holds. 

n n -. 

Proof. The statement follows from the Karamata Majorization Inequality. The inequality claims 
that for every two non-increasing sequences that satisfy (2) and for every increasing convex function 

/, 

n n 
i=n—k+l i=n—k+l 

Plugging in f{x) = (defined on (0,u)), we obtain the desired inequality. □ 

Lemma 3.8. Let A be a positive semidefinite matrix such that A ^ /„. Assume Tr(^) < r G N. 
Then for every positive semidefinite matrix M , A» M < X^^^^v-r+i 
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Proof. By von Neumann's inequality [14], A»M = tr(^M) < XlHi Aj(A)Ai(M). Since J27=i H^) < 
r and all Xi{A) < 1, we can easily see that the above product achieves its maximum when the largest 
r eigenvalues of A are 1 and the rest are 0. In this case, we have, A • M < Y17=i — 

As a corollary we get the following result. 

Corollary 3.9. Let X, M* and T be as in Theorem 3.3. Then for any positive semidefinite matrix 
U, we have U • {M* - X) < ElU-r+i >^i{U)- 

3.2 Barrier Shifts 

In this section, we analyze how we can update matrices A^^^ and B^'^\ and increment barriers 
I and r so that the upper and lower potentials do not increase. Let us think of $"(^4) as a 
function of an dimensional vector (consisting of entries of A). Then in the first approximation 
$«+5i7(-^ + tY) « $«+'5c/(A) + tY»U, where U is the gradient of $"+'^1/ at vl (C/ is an n x 
matrix). Thus the potential function does not increase, roughly when 

¥%4fS^^ - ^- Similarly, ^i+sAB + tY) < ^B), roughly when • ^—^^^-^ > 1, 
where L is the gradient of ^i+s^ at B. Following [3], we make these statements precise (we need 
to take into account lower order terms). We define matrices Ua and Lb, 



•siB-il + 6L)L)j 



Lemma 3.10. (Upper Barrier Shift) Suppose Xmax{A) < u and Y = vv'^ is a rank-one update. Lf 
Ua*Y <^ then + tY) < $«(A) and Xmax{A + tY)<u + Su. 

Proof. Let u' = u + 5u and P = PL{A+tY)- By the Sherman-Morrison formula (Lemma (3.4)), we 
can write the updated potential as: 

+ tY) = tr P(u7 -A-tY)-'P = trp( {u'L - A)-' + ^Wl-AT^YWl- A)-^ \ ^ 

\ 1 — t{wL — A) ^ •Y J 

= ivPiu'L - A)~'^P + 



N-iD tP{u'L - A)-^Y{u'L - A)-^P 



1 - t{u'L - y4)-i • Y 



<$ ^^^^) + i-t{u'L-A)-^.Y 

<$-^^(^)+ 'i-'^-^r^-y 



1 - t{u'L - ^)-i • Y 



$"(^) - ($"(y4) - $"+'^^(^4)) + ^ ^ 



l/t - {u'L - A)-^ m Y 



Here, we used Corollary 3.7 for the inequality on line 4. 

Substituting Ua»Y <l/t gives ^^+^u(^a + tY) < $"(^). The statement about Amax follows 
from continuity of eigenvalues. □ 
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Lemma 3.11. (Lower Barrier Shift) Suppose Ajnin(-B|^) > I + 6l and Y = vv^ is a rank-one 
update. IfLB»Y> l/t then ^i+s^B + tV) < ^i{B) and Xmin{{B + tY)\s) >1 + 6l. 

Proof. We proceed as in the proof for the upper potential. Let I' = I + Sl and P = Pg. By the 
Sherman-Morrison formula for the pseudoinverse (Lemma 3.5), we have: 

^l+5j^{B + tY) = tr{P{B + tY- l'l)P)^ = tr{P{B - l'l)P + tPYP)^ 

Ur((P(S - l'I)P)^Y{P{B - r/)P)t) 



tr(P(5 - i'i)py 



l+t{P{B - l'I)P)^ mY 

t{P{B - l'I)P)^^ . Y 



l + t{P{B -l'I)P)'! •Y 



Note that matrix Ua is positive semidefinite. Rearranging shows that ^i^s^iB + Y) < ^i{B) when 
-^^(Tr) > l/t. It is immediate that \aiiniPs{A + tTnr'^)Ps) > 1 + 6l since XminiPsAPs) > I + Sl. □ 

Now we prove that we can choose Yi and t so that conditions of both lemmas are satisfied. 

Lemma 3.12. (Both Barriers) //$"(^) < eu and ^i{B) < cl and eu,eL,Su,SL satisfy 

< ^ + e{7 + max(Ar,r) < -L _ e^, 

and X, Yi, costi, Z , T and N as in Theorem 3.3, M* — X is non-singular on S, then there exists 
i and positive t for which 

Lb • (ZYiZ) >l/t>UA» Yi, and (4) 

costi -t <l/iua^{N,T). (5) 

We will use the following lemma 
Lemma 3.13. Y:T=i UA*Yi<-^-h€u and YT=i Lb • {ZYiZ) > ^ - e^. 

Proof. 1. We use Corollary 3.9 to bound the Probenius product of Yi with each of the two summands 
in the definition of XJa (note that they are positive semidefinite), we get 

m 111 

^UA*Yi = UA*^Yi = UA*iM* - X) 

i=l i=l 

= tTA)-i'Al) ' - ^ - • - 

i=n-r+l \ V / V // i=n-T+l 



E" 1 n 
i=n-T+l (u+Su-\i{A))^ 1 

$«(^) - ^u+Su(A) ^ {U + Su) - Xi{A) 
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Note that the first term is at most l/(5[7, since 



n ^ n 

1 " ^ 1 1 \ _^''{A)-^''+^^{A) 



^^^^^\u - \iA) {u + Su) - Xi{A) y ou 
and the second term equals ^''+^u(^a). Thus X^i™! ^^yi • >i < e;/ + 

2. Let P be the projection on Im(M* — X). Since (M* — X) is non-singular on S, PPs = Ps- We 
have, 

m m 

Y^Lb* ZYiZ = Lb»Y^ ZYiZ = Lb» Z{M* - X)Z = Lb»P 
i=l 1=1 

f {Ps{B-{l + SL)l)Ps)^^ .p,„ r/^AU^P^t 



^i+sAB) - MB) ^ Xi{B\s) -{1 + Sl) 
> 1/Sl - eL, 
where the last line follows from Claim 3.6 in [3]. 

□ 

Proof. (Of Lemma 3.12) For the previous lemma, we get: ^i^i(t/yi • Yi + max{N,T)costi) < 
+ eu + muxiN.T) < Lb • {ZYiZ). Thus for some i, UA»Yi + mayi{N,T)costi < Lb • {ZYiZ). 
Letting t = {Lb • {ZYiZ))'^, we satisfy (4) and (5). □ 



3.3 Proof of Theorem 3.3 

Now we are ready to prove Theorem 3.3. We assume that M* — X is non-singular on S (which we 
can ensure by an arbitrary small pertrubation). 

We start with A^^) = X, = and aU weights wf*^ = 0. We define parameters as follows, 

Sl = 1/(2 max(Ar, T)), €l = 1/(4(5l), lo = -^kSi, 

Su = 4(5l, eu = 1/(4(5l), uo = 4T5l + 1, 

so as to satisfy conditions of Lemma 3.12, $"(A(°)) = $"(X) = Y^^i uo-K+i-iiX) ^ ^/{uo - 1) = 

e[7, $/(^^°^) = Ef=i 0^ = -k/lQ = et, l/5u + eu + ma^{N, T) = § max(iV, T) = 1/6l - eL- Then 
we iteratively apply Lemma 3.12. At iteration q, we find an index i and a positive t such that 
LQ(q){ZYiZ) > 1/t > U^(q){Yi), costi • t < 1/ max{N,T), and increment the weight of matrix Yi 

by t: wf^^^^ = w^f^ + t; update I = I + 5l and u = u + djj. The total cost increases by at most 
1/ max(A'', r). Finally, after N iterations we obtain matrices A^^^ and B^^^ with 

Amax(^^'^^) <uo + N6u = 2(iV + T)/max(iV,T) + 1 = ^^a. 
Xrain{B^^^\g) > Iq + N6l = {N/2 - 2fe)/ max(iV, T) = 0^^^. 
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Now consider an arbitrary unit vector v. Let v = vs + t'gx, where vs & S and Vg± ± S. Since 
B^^^ y e^inPs and vs e S, 

v'^A^^'^vs = vl{X + {Ps{M* - X)Ps)'/^B(''\Ps{M* - X)Ps)'/^)vs 
> vl{X + {PsiM* - X)Ps)^/^e^^PsiPsiM* - X)Psf/-')vs 

= OminVsM*Vs + (1 - ^min)4^^5 > ^minAmin(M*) ||i;5 f . 

On the other hand, Vs-L < 6'max|b5±||- Thus from the triangle inequality for the norm 

induced by A^^\ we get 

> elllX^,^{M)y^vs\\-eUl\\vs.\\ > Omin(M)V2-(C4l+^^f,A^i„(M)V2)||.^x||. 
On the other hand, since S is an eigenspace of X corresponding to k smallest eigenvalues. 

One of the two bounds above for (v-^A^^^v)^^^ increases and the other decreases as \\vg±\\ in- 

A*i/2+eV2^+0V2A^i,(M*)i/2 • 



creases. They are equal when \\v<^±\\ = ^Tl^^'i/f -• Therefore, (v'^ A^^^v)'^/^ > 



/)l/2 ^*l/2^ /j,yr*\l/2 

1/0 1/2 Br r- We conclude that 



A„.in(^(^)) = min > O^^X*X^UM*) 



Plugging in the values of parameters, we get the statement of the theorem for M = A^^\ The 
total cost is at most N/ max{N,T) = mm(l, N/T). □ 

Finally, wc prove Claim 3.2. 

Claim 3.2. Let V = hn^Ca+w) = ker(£(3+vK)"'"- Let £e be the Laplacian of the edge e. Define 



Yf. = We 



V 



Since Cq + YJ^&Ew '^^^^ ^ ^^G+w, we have X + Y^eaEw ^ ^- definition of the {k, T, X*)- 

patch, tr(7 - X) < T and A* < X''+'^{X). We apply Theorem 3.3 to matrices X, Ye and M* = I. 
We obtain a set of weights pe — supported on at most N edges — such that 

ci min(iV/r, 1) • A* < X^in [x + Y^e^Ew '^'^O " ^"^"'^ ^ ^eei?w '^^^O - 
Let lie = PeWe- Weights Wi define subgraph with at most N edges. It follows that 

ci imn{N/T, 1)X*Cg+w ^ ^G+Wk ^ C2>Cg+W'- 

The total weight of edges of Wk is Eee^vK ^^'^^ = i^eeEw Pecoste) T^deEw '^d < min(l, N/T) J2deEw "^d- 

□ 
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4 Constructing Nearly-Optimal Ultrasparsifiers 



We now apply our subgraph sparsification to build ultrasparsifiers. Recall that a weighted graph U 
is a (k, A;)-ultrasparsifier of another graph Gift/ :< G :< k-U and U has only n — l + k edges, where 
n is the number of vertices in U and G. The main result of this section is the following theorem. 

Theorem 4.1. For any integer k > 0, every graph has an log nO{loglogn),k)-ultrasparsifier. 

Our basic idea to build a good ultrasparsifier U is quite simple. Without loss of generality, we 
can assume that G is connected and has 0(n) edges. Otherwise given a graph G, we can first find a 
linear size sparsifier using [3], for each of its connected components, and build a good ultrasparsifier 
for each component. Because U is only k edges aways from a tree, our construction starts with 
good tree T. As it will be much more clear below, the quality of a tree is measured by its stretch, 
as introduced by Alon, Karp, Peleg and West [2]. 

Suppose T is a spanning tree of G = {V,E,w). For any edge e G E, let ei,-- - , G F be 
the edges on the unique path in T connecting the endpoints of e. The stretch of e w.r.t. T is 
given by st7-(e) = ■w(e)(^*Lj^ w^^' '^^^ stretch of the graph G with respect to T is defined by 
stT(G) = J2e£E^^T{e)- Our construction will start with a spanning tree with the lowest possible 
stretch. By [1], we can in polynomial time grow a spanning tree T with 

stT{G) = 0(n log n log log n(log log log n)^). 

Remark 4.2. For the sake of simplicity of the presentation, we will show the construction of 
ultrasparsifiers with Q(k) edges. We note that by choosing the appropriate constants, the number 
of edges can be made exactly k. 

Let K = ci ■ str(G)/fc for a sufficiently large constant ci. Our job is to choose Q(k) more 
weighted edges W and set U = T + W such that C2-U ^ G ^ k-U, for a constant C2. To this end, 
let W = (1/(c3k)) • G, for some constant C3. Then, G = C3K ■ W ^ c^k ■ {W + T). Also, because 
T ^ G, we have T + W ^ (1 + l/(c3'«))G ^ C4 • G, for a constant C4. Therefore, if we can find a 
e(A:)-edge subgraph W oiW such that T + W ^ 6(1) • (T + VF), we can then build a n - 1 + e{k) 
edge graph U = T + W satisfying C2 ■ U :< G ^ k ■ U (if we choose our constants q's carefully). 

To apply our subgraph sparsification results to construct W, we use the following structure 
result of Spielman and Woo ([21]: Theorem 2.1 and Corollary 2.2). 

Lemma 4.3. (Theorem 2.1 in [21]) (1) Tr{C\!'''^ CgC\^^'^) = sir(G). (2) For every t > 0, the 
number of eigenvalues of ^ CqC^j^ ^ greater than t is at most stT(G)/t. 

We now use Lemma 4.3 to prove the following lemma, from which Theorem 4.1 follows directly. 

Lemma 4.4. W is a {k,0{k),@{l)) -patch forT. 

Proof. Let Aj = \i{{Clj,j^^^Y/'^CT{Clp_^^Y/'^) be the i-th. eigenvalue, and yi be the corresponding 

1/2 

eigenvector. Let Xi = Lj,j^^y^. Then, 

\i = \i{{C^ f/'^CriC^ f/'^) = ^I^TXj ^ xfCxXj ^ 

* ' ^"^^ '^'^^ xf CrXi + xf CwXi xfCTXi + xfCaXi/ic^K)' 
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implying 



xJCaXi 1 - Aj stT(G) stT(G) 



xf CrXi Xi \ Xi J k 



C1C3 1— Ai 



It follows from the definition of A, that < Aj < 1. Hence, (1 — Ai_i)/Aj_i > (1 — Ai)/Aj. By 
Courant — Fischer theorem and the property 2 of Lemma 4.3, we have k < ^j^^- Therefore, 
Afe+i > = 9(1). We also have, 

tr ((4^^) 1/2^:^(4^^)1/2) < tr ((4) V2/:^(4)i/2) = _L tr ((4) V2£^(4)i/2) 

< ^^^MG) = — = @{k). 

- C3CiStT(G) ^ C3C1 ^ ^ 

Wc proved that is a {k, 0{k), e(l))-patch for T. □ 

We next show that the parameters of the ultrasparsifiers we obtained are optimal, up to low 
order terms. 

Theorem 4.5. Let G be a Ramanujan d-regular expander graph, for some constant d. Let U a 

(k, A^) ultrasparsifier for G. Then k > ^ logn. 

Proof. Let T be a low-stretch spanning tree of G, as above. As mentioned in [1], stT(G) = 
ri(mlogn) where m is the number of edges of the original graph. Prom lemma 4.3, and the 
conditions on the stretch of T we have Tr{CGJO-T^) = striG) > G ■ nlogn for some constant G. 

Since x'^ Cqx = ©(1) for the expander, the above inequality implies that X^ILi x'^ ' Ctx — '^log" 
where Xi are the eigenvectors of Cg{C-t)'^ . It is immediate from Markov's inequality that there exists 
some k such that xy?^ Ctx^ < ^1^^^ ■ Assume that for all i < wc have xp" CtXi < x^^ CrXk < 
j^^^^ ■ (Otherwise take k' < k appropriately). Then also XkiCx) < ;nogn' minmax theorem 

for eigenvalues this implies that adding N = k — 2 edges to T will result to a graph U with 
A2(>C[7) < Afc(£r) < riiogn ' '^^us any ultrasparsifier U with N edges will have 

G2 = \2{L,g) < kX2{jCu) < — j ^ K> n{ ) = i2( -— ) 

n log n k Jy 

□ 



5 Maximizing Algebraic Connectivity by Adding few edges 

In this section, we present an approximation algorithm for the following problem: given a graph 
G = {V,Ei,ase): ^ sct of Candidate edges Ecand: and a parameter k, add at most k candidate edges 
to G so as to maximize its algebraic connectivity, that is, find a subset E C Ecand that maximizes 
X2{£^G+e)- The problem was introduced by Ghosh and Boyd [5], who presented a heuristic for it. 
It is known that the problem is NP-hard [15]. But prior to this work, no approximation algorithm 
was known for it. 

We use two upper bounds for the cost of the combinatorial solution in order to prove an 
approximation guarantee: one upper bound is the SDP value, Xsdp, and the other is Xk^2{^G) 
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(see Lemma 5.1). Note that neither of these two bounds are good approximations for the value of 
the optimum solution by themselves (for instance, if G consists of n isolated vertices, {V, Ecand) is 
an expander, k < n, then the value of the combinatorial solution is but Xsdp ~ k/n), but their 
combinations lead to a good upper bound for the optimum solution Xqpt- 

For clarity and simplicity of exposition, wc assume here that (V, Ei^ase) and (V, Ecand) are 
bounded degree graphs with the maximum degree A. Our algorithm uses a natural semidefinite 
relaxation that was also used by Ghosh and Boyd [5] . We introduce a variable We (the weight of the 
edge e) for each candidate edge e G Ecand', add constraints that all edge weights are between and 1, 
and the total weight is at most k. Then wc require that A2(>CG + X^e ^e-Ce) > Xsdp (where Ce is the 
Laplacian of the edge e). We do that by adding an SDP constraint /^G + Se^e-^e ^ XsDpP(i,...^i)-'-, 
where is the projection on the space orthogonal to (1, ... , 1)"*-. We get the following SDP 

relaxation. 

maximize: Xsdp, 

subject to: Cg + ^ We^^e ^ Xsdp ■ -P(i,...,i)-l, 

eeE 

cand 

^ We<k, 

< fWe < 1 for every e G Ecand- 

We solve the semidefinite program and obtain solution {we}eeEcand- '^^^ total weight of all edges 
is k, however, the number of edges involved, or the support of the solution could be significantly 
higher than k. 

We use our algorithm to sparsify the SDP solution using Theorem 3.3. More precisely, we 
apply Theorem 3.3 with X = i2G'/(4A) and = WiCe/ i^^) restricted to the space (1, . . . , 1)"*-, 
N = 8k, T = tT{J2^WeCe) / {4:A) < k and cosU = Wi (we divide Cq and jCe by 4A to ensure that 
Xinax{X + Yle ^) — -'-)■ ^ weights Pe Supported on at most 8k edges s.t. 

— X2{Cg+'^ PeWeCe) = Amin(^+^ Pe>"e) > cAfc+2(X)Amin(^+^ l^e) > C ^ Xk+2{^G)XsDP- 
e e e ^ ' 

That is, we obtain a combinatorial weighted solution We = PiWi whose value is at least cXk+2{^G)XsDp/ i^^) 
(if A; + 2 > n, the value is at least cXsdp)- We next show that Xsdp > Xqpt and Afe+2(G) > Xqpt- 
Therefore, the value of the solution is at least cXQprp/A. 

Lemma 5.1. The value of the optimal solution, Xqpt, is at most Xk+2{^G)- 

Proof. Consider the optimal solution E. Let >Ce be the Laplacian of the graph formed by E. 
Note that rank(/l£;) < \E\ < k, therefore, dimker^e > n — k. Let S be the k + 1-dimensional 
space spanned by the eigenvectors of Cq corresponding to X2{Cg)-, • • • , Xk+2{t^G)- Since dimS" + 
dimkeri? > n, spaces S and ker^g have a non-trivial intersection. Choose a unit vector v G 
kerSTl^E. We have v{Cg + £'e)v^ < Afe+2(>CG) + = X}.^2{J^g)- Also v is orthogonal to the vector 
(1, . . . , 1)^. Therefore, Xqpt = X2{Cg + Ce) < Xk+2{j0-G)- □ 

The edges in the support ofwe, E = {we ■ 7^ 0}, form a non- weighted combinatorial solution. 
Since Amax('Cx + ^^WeJC-e) = 0(A), all weights We are bounded by 0(A), and thus the algebraic 
connectivity of G + is at least cXk^2{CG)XsDP / ■ 
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Theorem 5.2. There is a polynomial time approximation algorithm, that finds a solution of value 
at least cA^p^/A supported on at most 8k edges with total weight at most k. If k > n the algorithm 
finds a constant factor approximation. 

We present two corollaries for special instances of the problem. 

Corollary 5.3. // it is possible to make G an expander by adding k edges (and thus Xqpt ~ 
then the algorithm finds a constant factor approximation. 

Note that if the graph formed by candidate edges is an expander then the value of the following 
SDP solution = k/\Ecand\ each edge e G Ecand is ^{k/n), thus Xsdp > ck/n. 

Corollary 5.4. // the graph formed by candidate edges is an expander, then the approximation 
algorithm from Theorem 5.2 finds a solution of value at least C:^Xopt- 

Remark 5.5. It is possible to get rid of the dependence on A in Theorem 5.2 and Corollary 5.4 
and obtain approximation guarantees of cm.m{XopT, Xqpj,) and ^Xqpt respectively. We omit the 
details in this extended abstract. 
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